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Abstract 

A tight upper bound is given involving the maximum of a supermartingale. Specifically, it is 
shown that if Y is a semimartingale with initial value zero and quadratic variation process [Y, Y] 
such that Y + [Y, Y] is a supermartingale, then the probability the maximum of Y is greater 
than or equal to a positive constant a is less than or equal to 1/(1 + a). The proof is inspired 
by dynamic programming. Complements and extensions are also given. 

1 Introduction 

The basic framework of the classical martingale calculus will be used, as described, for example, 
in [1, 2, 5, 6, 7]. All random processes are assumed to be defined on a complete probability space 
(Q,J-,P) with a filtration of u-algebras [Tt : t > 0) that is assumed to satisfy the usual conditions 
of right-continuity and inclusion of all sets of probability zero. Semimartingales are processes that 
are cadlag (right continuous with finite left limits) and can be represented as the sum of a cadlag 
local martingale and a cadlag, adapted process of locally finite variation. The quadratic variation 
process of a seminartingale Y, denoted by [Y, Y], is the nondecreasing process defined by 

kfi 1 

[Y,Y] t = lim ^(it ?+1 -%) 2 

i=0 

for any choice of = < < ■ ■ ■ < tJJ = t such that max, — tf\ — » as n — > oo. The process 
[Y, Y] can be decomposed as [Y, Y] t = ^ s<( (Ay s ) 2 + [Y, Y]%, where AY S is the jump of Y at s, and 
[Y, Y] c is the continuous component of [Y, Y\. 

Recall that a process Y is a supermartingale if E[Yt — Y S \J- S ] < for < s < t. Intuitively, 
it means that the process has a downward drift. The following condition is stronger than the 
supermartingale condition, requiring that the downward drift be at least as strong as a constant 7 
times the rate of variation of the process, as measured by the quadratic variation process. 

Condition 1.1 The process Y is a semimartingale with Yq = 0, and 7 > such that Y + j[Y,Y] 
is a supermartingale. 

Condition 1.1 is a joint condition on Y and [X,Y], and is preserved under continuous, adapted 
time changes. A stronger condition, involving separate constraints on Y and [Y, Y], is the following. 
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Condition 1.2 Y is a random process with Yq = 0, and 7 > 0, fi > and a 2 > are such that 
7 = -jji an d both (Y t + fit : t > 0) and ([Y, Y] — a 2 t : t > 0) are supermartingales. 

Condition 1.2 implies Condition 1.1 because Y t + j[Y, Y] t = (Y t + fit) + j([Y, Y] t - a 2 t). 
Let Y* = sup{Y t : t > 0}. 

Proposition 1.1 Suppose Y and 7 > satisfy Condition 1.1 (or Condition 1.2) and a > 0. 

(a) The following holds: 

HY-^i^- (1) 

(b) Equality holds in (1) if and only if the following is true, with T = inf{i > : Y t > a} : 
(YtAT '■ t > 0) has no continuous martingale component, Y is sample- continuous over [0, T) with 
probability one, P{Yr = 0|T < 00} = 1, and (Y + j[Y, Y]) tA x is a, martingale. 

Proposition 1.1 raises the question of whether there is a finite upper bound on i£[Y*] depending 
only on 7. A related question is the following. The necessary conditions in Proposition 1.1(b) 
cannot be satisfied for two distinct strictly positive values of a. This raises the question as to how 
close to equality the bound in part (a) could be for all values of a, for a single choice of Y not 
depending on a. The following proposition addresses these two questions. 

Proposition 1.2 Given 7 > there exists Y satisfying Condition 1.2 such that 

P{Y* > a}> ——^ (2) 

1 " S ~ 5(l + a7) V ' 

for all a > 0. In particular, E\Y*\ = +00 for this choice ofY. 

Remark 1.1 This remark focuses on the case that Y has stationary, independent increments 
(SII) and Y and 7 satisfy Condition 1.2. The moment upper bound of Kingman [3, 4] f or the 
waiting time in a GI/G/1 queue implies 1 

E[Y*] < — , (under SII) (3) 
27 

which together with Markov's inequality implies that 

P{Y* >a}<^-. (under SII) 

The remainder of this paper is organized as follows. Section 2 describes a construction, showing 
there is process meeting the conditions of Proposition 1.1(b), and providing a proof of Proposition 
1.2. Proposition 1.1 is reformulated and proved in Section 3. A discrete time version of Proposition 
1.1 is stated and proved in Section 4. Discussion about intuition and possible extensions is given 
in Section 5. 



1 Kingman's moment bound is for discrete time SII processes, but it is easily extended to continuous time by 
sampling the continuous time processes at times of the form 2 _n , and letting n — > 00. 
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2 The big jump construction 



A construction is given for a process meeting the bound of Proposition 1.1 with equality, and for a 
process providing a proof of Proposition 1.2. These processes satisfy Condition 1.2. 

Let fi > 0, a 2 > 0, and h : R + — > M + such that /i is nondecreasing. Consider a random process 
Y of the form: 

-y(t) t < T 

%(T))- M (t-T) t>T 



for a deterministic, continuous function y = (y(t) : t > 0) and random variable T described below. 
Note that if T < +oo then Y T _ = -y(T), Y T = h(y(T)), and AY T = y(T) + h{y{T)). Let y be a 
solution to the differential equation 

and let T be an extended nonnegative random variable such that for all t > 0, 

^2 



P(T > t) = exp(— / n(y s )ds) with n(y) 
Jo 



(y + My)) 2 ' 



The function n(y(t)) is the failure rate function of T: P(T < t + r?|T > t) = n(y(t))r) + o(r?). The 
function k was chosen so that 

E[(Y t+v -Y t ) 2 \T>t] = (y(t)+h(y(t))) 2 K(y(t)) V + o( V ) 

= a 2 r] + o(rj) 

and the differential equation for y was chosen so that 

E[Y t+v -Y t \T>t] = -mv + {y(t) + h(y(t))K(y{t))r, + o{r]) 
= -fir] + o(r]) 

Therefore, Y satisfies Condition 1.2. 

If the function h is strictly increasing, then for any c > 0, a change of variable of integration 
from t to y yields: 

P{Y*>h(c)} = P{T > y~\c)} -P{T = oo} 

= exp(-/(c)) - exp(-/(oo)) (4) 

where 

r» _1 (c) 



1(c) = / K(y(t))dt 
Jo 

= [ K (y) 

Jo 



a 2 



" + ^T%)} dy 
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Example 1: Meeting Proposition 1.1 with equality. Take h(y) = a for some a > 0. We 
don't use (4) because h is not strictly increasing, but similar reasoning yields: 



P{Y* >a} = 1 - P{T = oo} 

= l-exp(-J K(y(t))dt\ 



1-exp K{y) ^+ — \ dy 



a 2 



-i 



oc 



= 1-expl-/ ^— M - 2 }> '/// 

1 



1 + ^ 

Thus, the process 1" satisfies the bound of Proposition 1.1 with equality for 7 



Example 2: Proof of Proposition 1.2. Take h(y) = b + y for some 6 > 0. Equation (4) yields 
that for b > 0, 

P{Y* > 6 + c} = exp(-/(c)) - exp(-/(oo)) 



where 



/(c) = / 
Jo 



' a 2 



a 



(b + 2y) 2 r b + 2y 



Jo {b + 2y 



fi(b + 2y)+a 2 



dy 
dy 



= -In 



6 + 2c 



Using this and (4) , and setting c = a — b, yields 



P{Y* > 



*>a}={ (^) 2 {( 1 + ^^) 2 



1} a>b 

0<a<b 



(5) 



Let b = Then P{Y* > a} = | > ^j^w^ for < a < 6. By checking derivatives, it is easy to 
verify that (1 + §)a — 1 > 4 ^ +a ^ for any a > 0. Therefore, for this choice of b, and a > b, 

4 



P{Y*>a} > - 1 + — -1 



(7 



2/ia 



> 



5(1 + $Y 



This bound for the process Y proves Proposition 1.2. 
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3 Reformulation and proof of Proposition 1.1 



Proposition 1.1 concerns the probability that a process with downward drift, starting from 0, reaches 
level a. It is more convenient for the proof, to consider the equivalent problem, of the probability 
a process with upward drift, starting from a, reaches zero. The correspondence between the two 
formulations is obtained by setting X = a — Y, so the proof of Proposition 3.1 below also establishes 
Proposition 1.1. 

Condition 3.1 X is a semimartingale and 7 > 0, such that X — ~/[X,X] is a submartingale. 

Proposition 3.1 (Equivalent to Proposition 1.1.) Suppose X and 7 satisfy Condition 3.1 and 
X = a for some a > 0. Let T = inf{/ : X t < 0} (so T = 00 if X t > for all T). 
(a) The following holds: 

P{T < 00} < — — . (6) 
1 1 ~ 1 + 7« 

(h) Equality holds in (6) if and only if (X^t '■ t > 0) has no continuous martingale compo- 
nent, X is sample- continuous over [0, T) with probability one, P{X? = 0|T < 00} = 1, and 
((X — j[X,X])tAT -t>0) is a martingale. 

Proof. (Proof of (a)) Suppose X and 7 satisfy Condition 3.1 and X$ = a for some a > 0. 
Let X t = max{Xt A T, 0}. Let D = —Xt on the event T < 00, and let D = otherwise. Note 
that X t = X t and [X,X] t = [X,X] t for t G [0,T), while, if {T < 00}, = X T > X T = -D and 
A[X,X] T = (X T _) 2 < (D + X T „) 2 = A[X,X] T . Thus, 

(X - 7 [X, X]) t - (X - 7 [X, X]) tAT = { D + i{2DX l_ + D 2 ) >v]i T <T ^ 

Since X — j[X,X] is a submartingale, so is (X — j[X,X]) t AT, and therefore so is X — j[X, X]. 
Therefore, X and 7 also satisfy the conditions of the proposition, and T = inf{i > : X < 0}. 
Therefore, to prove (a), it suffices to prove (a) with X replaced by X. Equivalently, in proving (a), 
we can, and do, make the assumption, without loss of generality, that X t = for t > T. 

Let p(x) = for x > 0. Let < s < t. By the Dolean-Dade Meyer change of variables 

formula for semimartingales, 

p(X t )=p(X s )+ f p'{X u „)dX u + Y, (p( X u) - P(Xu-) - p'(X u _)AX u ) + \ f p"(X u )d[X,Xf u 

Js s<u<t 1 Js 

(8) 

Observe that 

p'(X u _) = -7P 2 (X U ~) (9) 

p(X u )-p(X u „)-p'(X u _)AX u = (AX u ) 2 j 2 p 2 (X u ^p(X u ) 

< (AX U ) 2 7 V(^«-) (10) 



and 



p"(X u _) = 2 1 2 p s (X u -) 

< 2 7 V(X U _). (11) 
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Combining (8) - (11) and the fact [X,X] U = [X,X] C U + ^„ <M (A1„) 2 



yields 



p(X t ) < p(X s ) - 7 ! p{X u ^) 



2 dG 



u 



(12) 



J s 



where G = X — ^\X,X\. By assumption, G is a submartingale, so 



E 




(13) 



s 



Therefore, p(X) is a supermartingale, so that E\p(Xt)] < p(Xo) = p(a). For t > 0, {T < t} = 
{p(X t ) = 1}. Therefore, P{T <t}< E\p(X t )] < p(a) for all t. Thus, P{T < oo} = lim*-^ P{T < 
t} < p(a), completing the proof of part (a). 

(Proof of (b)) To begin, assume that X t = for t > T, with probability one. Note that there 
are three places in the proof of part (a) where the bounds might not hold with equality. These are 
associated with inequalities (10), (11), and (13). First, if AX S ^ 0, then (10) holds with equality 
if and only if X s ^ 0. Therefore, in order for (6) to be tight, with probability one, any jumps of 
X happening in [0, T] must happen at time T. Second, the inequality (11) for p"(X u ^) is strict for 
any u < T. This inequality is applied to the last term in (8). Therefore, the resulting upper bound 
on the last term in (8) holds with equality if and only if [X, X] c = over the interval (s, t\. Finally, 
(13) holds with equality if and only if G is a martingale (not just a supermartingale) over (s,t\. 
These observations prove (b) under the additional assumption that X t = for t > T. 

To prove (b) in general, suppose X satisfies the conditions of the proposition, and let X be 
defined in part (a). The conditions in part (b) involve X only up to time T. The process X and 
X are equal for t < T, so the only difference between them over [0, T] could possibly be at time T. 
If (6) holds with equality, then by the special case of part (b) already proved, X must satisfy the 
conditions in (b). In particular, (X — j[X, X])tAT is a martingale. In view of (7), (X — j[X, X]) tA T 
must also be a martingale and P{D = 0} = 1, or equivalently, P{Xt = 0\T < oo} = 1. Therefore, 
X also satisfies the conditions in (b), and the proof of the proposition is complete. H 

The above proof is inspired by dynamic programming. The problem of finding X to maximize 
P{T < oo} subject to the given constraints can be viewed as a stochastic control problem. In- 
tuitively speaking, given X s = x for some time s and x > 0, the conditional distribution of the 
increment X s+V — X s must be selected subject to first and second moment constraints. The function 
p plays the role of the value function in dynamic programming. 

4 Discrete time processes 

For the discrete-time setup suppose that (f2, J 7 , P) is a complete probability space with a filtration 
of cr-algebras {Tk '■ k G Z + ). 

Condition 4.1 S = (Sk : k G Z + ) is an adapted random process with Sq = and, with Uj = 
~ "Sj'-i f or 3 > 1; {Sk + 7 J2i<j<k Uj:k> 0) is a supermartingale. 



Let S* = sup{Sk ■ k £ Z + }. 
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Proposition 4.1 Suppose S and 7 > satisfy Condition 4-1 and a > 0. 

(a) The following holds: 

P{S* >a}< — - — (14) 
1 + 7« 

(b) For any 7 > 0, a > and e > 0, there is a process S satisfying Condition 4-1 such that 
P{S* > a} > ^ - e. 

Proof. (Proof of (a)) The filtration (jF fc : k € Z + ) can be extended to a nitration [Tt '■ t E M_|_) 
by letting J-j = T\t^ for i € R + , and the process S can be extended to a piecewise constant process 
(Yf.te R+) by letting Y t = for t e M+. Then S 1 * = Y* and Y satisfies Condtion 1.1. Thus, 
by Proposition 1.1, P{S* < a} = P{Y* < a} < This establishes (a). 

(Proof of (b)) If 7 = 0, the process S can be a mean zero random walk with finite variance 
jumps, and then S* = 00 with probability one, so that equality holds in (14) for such choices of S. So 
assume for the remainder of the proof that 7 > 0. Given fi > 0, let a 2 = ^ — fi 2 . Assume that fi is so 
small that a 2 > 0. Let a = a + fi. Let Y be a continuous time martingale as constructed in Example 
1, for the parameters fl,a 2 , and a. Thus, Y follows a deterministic trajectory up to some random 
time T, which could be infinite. If T < 00, then Y jumps up to a at time T. Furthermore, modify 
Y on the event {T < 00} by letting Y t = a-fi(t- T) for t > T. Then Y t - fit and [Y, Y] - d 2 (t A T) 
are martingales, and P{Y* > a} = , where 7 = Let M denote the martingale defined by 
Mi = Y t — fit. Since M differs from Y by a continuous function with finite variation, [M, M] = [Y, Y] 
with probability one. Since M is a martingale with finite second moments, M 2 — [M.M] is also a 
martingale. 

Let S = (Sk ■ k £ Z + ) be obtained by sampling Y at nonnegative integer times, namely, = Y& 
for k G Z + . We first check that 5 satisfies Condition 4.1. For fc > 1, set Uk = Sk — Sk-i = Y^ — Y^-i. 
Then E[U k \Fk-i] = E[Y k - r fc _i|^ fc _i] = fi, and 

£?[t/ fc 2 |^fc-i] = ^[(A + Mfc-Mfc.x) 2 !^.!] 
= fi 2 + E[(M fc - M w ) 2 |f w ] 
= /x 2 + ^[[M,M] fc -[M,M] fc _ 1 |^ fe _ 1 ] 
= + X] fe - [X, X] fc _! |^ fc _i] < /i 2 + a 2 

Therefore, by the choice of a 2 , 

E[U k - lUt^k^] > A - 7(A 2 + <7 2 ) = 

Thus, S satisfies Condition 4.1 as required. On the event {T < 00}, Yt = a = a+pL and Y decreases 
with constant slope —ft after time T, implying that 5m = Y^ > a, so that S* > a. Therefore 

P[S* >a}> P[Y* >a} = — l — = = (15) 



1 + 7« 1 + 



[A + °] 



For fi sufficiently small, (15) implies the inequality of part (b), and the proof is complete. 

5 Discussion 

The proof of Proposition 1.1 might leave the reader wanting some additional insight into why the 
big jump processes are the ones meeting the bound with equality. As noted in Section 3, it is useful 
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to view the bound p(a) = 1+ ^ a as a value function in the sense of dynamic programming. Given its 
explicit form, it is trivial to verify that it is decreasing, convex, and the third derivative is negative. 
However, one might expect these properties even without knowing the function explicitly. The fact 
it is decreasing means that the chances of X ever reaching zero decrease with the initial state a. 
Due to Jensen's inequality, the fact the function is convex means that the chances of ever reaching 
zero from an initial state a would increase if, before imposing the constraints on X, the process 
could follow a martingale trajectory. Finally, the fact that the third derivative is negative means 
that the second derivative is decreasing. Therefore, it is better to use the variations of X at smaller 
values. In this connection, it is useful to remember that martingales, even discontinuous ones, can 
be obtained from time changes of brownian motion. That explains why the process, for initial state 
a, has only two possible values at future times, and why there are no upward jumps in the process. 

We close by speculating about two possible extensions. Proposition 1.1 shows that, for continuous- 
time processes, even though Condition 1.2 is more restrictive than Condition 1.1, the same maxi- 
mum value of P{Y* > a} can be achieved. However, the situation is different in discrete-time. The 
discrete-time version of Condition 1.2 would constrain the process Y to take downward jumps that 
are not vanishingly small, and the maximum possible value of P{Y* > a} would be strictly smaller 
than the bound in Proposition 4.1. The intuition in the first paragraph above would indicate that in 
such a case, the optimal process would still be one following a deterministic path, with the possible 
exception of a single big jump, reaching the target value in one step. 

Proposition 1.2 shows that under Condition 1.1 it is possible that £7[Y*] = +oo. However, it 
might be interesting to consider maximizing the expected value of sublinear functions of Y, such 
as .E[(Y*) a ] for a fixed a with < a < 1. The process of Proposition 1.2 achieves at least one-fifth 
of the maximum possible value. Perhaps the big jump construction of Section 2 still yields the 
extremal processes, for a suitable choice of the function h. 



References 

[1] J. Jacod. Calcul stochastique et problemes de martingales, Lecture Notes in Math., vol. 714- 
Springer- Verlag, New York, 1979. 

[2] O. Kallenberg. Foundations of Modern Probability (2nd ed.). Springer, 2002. 

[3] J.F.C. Kingman. Some inequalities for the queue GI/G/1. Biometrika, 49(3/4):315-324, De- 
cember 1962. 

[4] J.F.C. Kingman. A martingale inequality in the theory of queues. Cambridge Phiolos. Soc., 
59:359-361, 1964. 

[5] P.A. Meyer. Un cours sur les integrales stochastiques, pages 246-400. Springer- Verlag, New 
York, 1976. 

[6] P. Protter. Stochastic Integration and Differential Equations (2nd ed.). Springer, 2004. 

[7] E. Wong and B. Hajek. Stochastic processes in engineering systems. Springer- Verlag, New York, 
1985. 



8 



